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Description 

[0001] The present invention relates to an expression of template patterns of sign-language word used in the rec- 
ognition based on the matching between continuous sign-language patterns, in which sign-tanguage words are con- 
tinuously coupled, and to the template patterns of sign-language word. 

[0002] The present invention also relates to a continuous sign-language recognition apparatus which continuously 
recognizes sign-language words included in continuous sign-language patterns in a high accuracy and an input appa- 
ratus for inputting a command or the like to a computer in a form of a gesture previously defined to the command. 
[0003] Techniques related to the sign-language recognition have been proposed in "Hand Motion Recognition Ap- 
paratus and Sign Language Translation System" in JP-A-2-1 44675 (prior art example 1), "Hand Motion Recognition 
Method Using Neuro-computer" in JP-A- 3-1 86979 (prior art example 2), "Gesture Recognition using a Time-Series 
Network" in 43th National Conference of Institute of Information Processing Engineers of Japan, IS-3, 1992 (prior art 
example 3), and "Sign Language Interpretation using Continuous DP Matching" in the Human Interface Study Group 
in Institute of Information Processing Engineers of Japan, 44-12, 1992 (prior art example 4). 

[0004] The prior art examples 1 and 2 are intended to recognize finger spellings. The prior art example 1 utilizes 
colored gloves to detect a positional relation between fingers by an image recognition technique. The positional relation 
is matched with previously stored finger spelling patterns to recognize each finger spelling. The prior art example 2 
recognizes data on finger spellings inputted from data gloves through a multi layer neural network. The prior art ex- 
amples 3 and 4, in turn, are intended to recognize moving sign-language patterns or gestures. The prior art example 
3 determines whether a gesture is dynamic or static by detecting the first pattern of a dynamic gesture through a multi 
layer neural network. Then, a motion is recognized through a recurrent type neural network from a starting point at 
which the first pattern is detected. A static gesture in turn is recognized through the multi layer neural network, tn this 
manner, different recognition systems are used for dynamic gestures and static gestures. In the prior art example 4, 
sign-language word patterns inputted from data gloves are registered in the form of a dictionary of template pattern of 
sign-language, such that sign-language words are recognized by the continuous DP matching of sign-language sen- 
tence patterns inputted from the data gloves with the template patterns of sign-language word. The system employed 
in the prior art example 4 stores not only static patterns but also dynamic patterns as similar time-series patterns of 
feature vectors, and does not distinguish them also for a recognition process. 

[0005] The sign-language patterns may be classified into dynamic patterns including motions and static patterns 
without motions. It is therefore desirable that storing methods and recognition methods appropriate to the respective 
patterns are employed in view of the storage capacity of a pattern dictionary, recognition accuracy and recognition 
speed. The prior art example 3 distinguishes static patterns and dynamic patterns and utilizes different recognition 
techniques to the static and dynamic patterns, wherein the static/dynamic distinction is made simply on the whole 
pattern. However, in the actual sign-language, this is often the case that only one component pattern, for example, 
"position of hand" is dynamic, whereas other component patterns are static, so that the static/dynamic distinction on 
a whole pattern will result in a low efficiency. 

[0006] Also, the foregoing prior art stores a whole time-series vector pattern inputted from data gloves or the like as 
one pattern in the recognition of a sign-language pattern including motions. For this reason, ail parameters associated 
to a feature vector at each time are used to carry out the recognition. However, a sign-language pattern may be broken 
down into relatively independent component patterns such as "position of hand", "direction of palm°, "figure of hand", 
and so on. Further, all component patterns do not always characterize a sign-language pattern, but only part of the 
component patterns, in many cases, characterizes a sign-language pattern. For this reason, if a sign-language pattern 
having less important component patterns is to be recognized by the method of recognizing a sign-language pattern 
as a whole, the recognition accuracy may be degraded due to fluctuations of the component patterns. Furthermore, 
each component pattern does not necessarily change in synchronism with changes of other component patterns. On 
the contrary, in many cases the respective component patterns change at different times from each other. Therefore, 
the distinction only as to whether a whole sign-language word pattern is static or dynamic cannot attend to the degra- 
dation of the recognition accuracy caused by component pattern changes occurring at different timings. 
[0007] Furthermore, the above-mentioned prior art distinguishes whether a sign-language pattern is static or dynamic 
in the pattern recognition by detecting a starting pattern of a dynamic pattern. However, as the number of words to be 
recognized is increased, many words may include similar motions. Thus, the method of distinguishing whether a pattern 
is dynamic or static by detecting a starting pattern of a dynamic pattern may easily cause erroneous recognition and 
therefore is not a satisfactory method. 

[0008] In an article entitled "A Hand Gesture Recognition Method and its Application" by T Takashi et al published 
in Systems and Computers in Japan, Vol 23. No. 3, of 1 January 1 992, a continuous sign-language recognition system 
was disclosed in which hand gestures were input using a Data Glove. In the system discussed in that article, template 
patterns of sign language words were stored in a dictionary memory, so that sign language patterns received from that 
Data Glove could be compared with the template patterns, to recognise the sign language words. The sign language 
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words could then be displayed. 

[0009] It is desirable to provide a flexible and highly accurate sign-language recognition apparatus which is capable 
of attending to fluctuations and displacements in particular component patterns as well as dynamic sign-language 
patterns. 

[0010] We now propose to classify each of component patterns constituting a template pattern of sign-language 
word, such as "position of hand", "direction of palm", "figure of hand", and so on, info a static pattern and a dynamic 
pattern. Then, each of the component patterns is stored based on the classification, i.e., in the form of a feature vector 
representing the pattern if it is a static pattern or in the form of time-series pattern of a feature vector if it is a dynamic 
pattern. 

[0011] Also, we propose to employ different recognition methods depending on whether each component pattern 
constituting a template pattern of sign-language word is static or dynamic. Specifically, a static component pattern is 
recognized by matching the same with feature vectors, while a dynamic component pattern is recognized by matching 
the same with time-series vectors. In the matching, static patterns as well as dynamic patterns are all matched in 
parallel so as to determine from the matching results which pattern is most effective. 

[0012] According to a first aspect of the invention, there is provided a continuous sign-language recognition system 
comprising: 

means for converting a series of sign-language words into sign-language patterns which are time-series patterns 
of feature vectors and inputting said converted sign-language patterns; 
means for storing template patterns for sign -language words; 

pattern matching means for matching said input sign-language patterns with said template patterns of sign-lan- 
guage word, to recognize a sign-language word expressed in said input sign-language patterns; 
recognizing means responsive to said matching means for recognizing a sign-language word from the results of 
matching between the input sign-language pattern and the template patterns; and 
means for outputting the sign-language word recognized by said recognizing means; 

wherein: 

each of said template patterns for sign-language words includes a description for identifying whether each of 
gesture elements each constituting a sign-language word is a static pattern or a dynamic pattern, and a pattern 
of the gesture element expressed as a form at that is different depending upon whether the gesture element is a 
static pattern or a dynamic pattern; 

said pattern matching means, referring to said template patterns in said storing means, selecting a relevant one 
of different matching schemes in accordance with whether each gesture element in the template patterns is static 
or dynamic and performing a matching of said input sign language patterns with the pattern of each gesture element 
using the selected matching scheme, independently from the other gesture elements; and 
said sign-language word recognizing means integrating recognition results of the gesture elements each consti- 
tuting a sign-language word, based on a time relation with the recognition results. 

[001 3] According to a second aspect of the invention there is provided a continuous sign-language recognition system 
comprising: 

means for converting a series of sign-language words into sign-language patterns which are time-series patterns 
of feature vectors and inputting said sign-language patterns; 
pattern storage means for storing template patterns of sign-language morphemes- 
word store means for storing sign-language words each expressed as a combination of sign-language morphemes 
constituting a sign-language word; 

pattern matching means for matching said sign-language patterns with said template patterns of sign-language 
morphemes, in order to recognize sign-language morphemes expressed in said sign-language patterns; 
morpheme recognition means responsive to said pattern matching means for recognizing sign-language mor- 
phemes from the results of matching between one of said sign-language patterns and the template patterns of 
sign-language morphemes; 

word recognition means connected to said morpheme recognition means for recognizing a sign-language word 
by matching the recognized sign-language morphemes with a combination of sign-language morphemes consti- 
tuting a sign-language word; and 

means for outputting the sign-language word recognized by said word recognizing means; 

said template patterns stored in said sign-language morpheme template storing means including a description of 

identifying whether each of gesture elements constituting a sign-language morpheme is a static pattern or a dy- 
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namic pattern, and a pattern of the gesture element expressed as a format different depending upon whether the 
gesture element is a static pattern or a dynamic pattern; 

said pattern matching means matching the input sign-language patterns with the sign-language morpheme tem- 
plate patterns, referring to said template patterns in said pattern storage means, selecting a relevant one of different 
matching schemes in accordance with whether each gesture element in the template patterns is static or dynamic, 
and performing a matching of said sign language patterns with the pattern of each gesture element using the 
selected matching scheme, independently from the other gesture elements; and 

said morpheme recognition means integrating recognition results of the gesture elements each constituting a sign- 
language morpheme, based on a time relation with the recognition results to recognize the sign-language mor- 
pheme. 

[0014] According to a third aspect of the invention, there is provided an input system comprising: 

sign-language-input means for converting a series of gestures into gesture patterns which are time-series patterns 
of feature vectors and inputting the converted gesture patterns; 
storing means for storing template patterns of gestures; 

matching means for matching the input gesture patterns with the template patterns of the gesture stored in said 
storing means, in order to recognize a gesture expressed in the gesture pattern; 

gesture recognizing means responsive to said matching means for recognizing the gesture expressed in the ges- 
ture pattern, from the results of matching between the input gesture patterns and the template patterns; and 
means for performing a processing corresponding to the recognized gesture; 

each of said gesture template patterns stored in said template pattern storing means including a description of 
identifying whether each of gesture elements constituting a gesture is a static pattern or a dynamic pattern, and a 
pattern of the gesture element expressed as a format different depending upon whether the each gesture element 
is a static pattern or a dynamic pattern; 

said matching means for matching the input gesture patterns with the gesture template patterns, referring to said 
template patterns in said storing means, selecting a relevant one of different matching schemes in accordance 
with whether each of gesture elements in the template patterns is static or dynamic and performing a matching of 
said gesture patterns with the pattern of each gesture element using the selected matching scheme, independently 
from the other gesture elements; and 

said gesture recognizing means integrating recognition results of the gesture elements constituting a gesture, 
based on a time relation with the recognition results. 

[0015] With the present invention, it is possible to determine whether each of component patterns constituting a 
template pattern of sign-language word is static or dynamic, integrating the results based on a temporally overlapping 
portion of the component patterns so as to recognize a sign-language word. 

[0016] For making a dictionary of template pattern of sign-language word, the respective component patterns con- 
stituting a template pattern of sign-language word are classified into a static pattern and a dynamic pattern. A static 
pattern is expressed by a single feature vector representing the pattern to reduce the storage capacity required to store 
the template patterns of sign-language word and to permit efficient storage of them. 

[0017] Each of constituent component patterns in the template patterns of sign-language word is classified into a 
static pattern and a dynamic pattern, and different recognition methods are employed for the static component pattern 
and the dynamic component pattern, respectively, whereby a recognition time can be made shorter than a recognition 
process using a single method for matching a whole pattern. 

[0018] Further, the component patterns of a sign-language word are separately matched with the corresponding 
component patterns constituting each template pattern of sign-language word, and the matching results are integrated 
to recognize the sign-language word, thus flexibly attending to a degradation in the recognition accuracy due to dis- 
placements in the respective component patterns and largely fluctuating component patterns. Also, since the most 
effective pattern is determined after all static and dynamic patterns are simultaneously matched, erroneous static/ 
dynamic pattern recognition is largely reduced, with the result that the recognition accuracy can be improved for the 
sign-language words. 

[0019] By applying the method of recognizing sign-language words as described above to an input apparatus for a 
computer, user's instructions, i.e., symbols and commands can be inputted to a computer by previously defined ges- 
tures. 

Fig. 1 is a block diagram showing the configuration of a recognition part 24 of sign-language word; 
Fig. 2 is a block diagram showing a system for realizing the continuous sign-language recognition- 
Fig. 3 is a table showing a format of template pattern of sign-language word stored in a dictionary 23 of template 
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pattern of sign-language word; 

Fig. 4A shows a sign-language word representing "people"; 

Fig. 4B shows patterns of the sign-language word "people" converted by data gloves 21 ; 

Fig. 4C shows patterns of the sign-language word "people" which is stored in the dictionary 23 of template pattern 
of sign-language word; 

Fig. 5 is a block diagram showing the configuration of pattern matching parts 11a, 11b, 11c; 
Fig. 6 shows graphs representing exemplary matching results of respective component patterns; 
Fig. 7 is a block diagram showing the configuration of a detection part 56 of static pattern; 
Fig. 8 is a block diagram showing the configuration of a detection part 55 of dynamic pattern; 
Fig. 9 is a block diagram showing the configuration of a detection part 12 of sign-language word; 
Fig. 10 schematically shows how component patterns overlap with each other; 

Figs. 1 1 A, 1 1 B schematically show an overlap of component patterns when all the component patterns are static, 

wherein Fig. 11 A is an example of the component patterns which are regarded as being overlapping, and Fig. 11 B 

is an example of the component patterns which are regarded as not being overlapping; 

Fig. 1 2 shows an example of a sign-language word expressed by a simultaneous gesture; 

Fig. 1 3 is a block diagram showing the configuration of a detection part 96 of simultaneous gesture; 

Fig. 14 is a table showing a format of a dictionary 23 of template pattern of sign-language which is additionally 

provided with descriptions of context dependent patterns as additional information; 

Fig. 15 is a block diagram showing the configuration of a detection part 12 of sign-language word which is addi- 
tionally provided with a means for extracting additional information from a context dependent pattern; 
Fig. 16 is a flow chart showing a procedure of a process of sign-language word; 
Fig. 17 is a flow chart showing a procedure of processes 162a, 162b, 162c of pattern matching; 
Fig. 18 is a flow chart showing a procedure of processes 163a, 163b, 163c of pattern detection; 
Fig. 19 is a flow chart showing a procedure of a process 64 of detection of candidate of word; 
Fig. 20 is a flow chart showing a procedure of a process 199 of detection of simultaneous gesture; 
Fig. 21 is a block diagram showing a system for realizing the continuous sign-language recognition when a dic- 
tionary of morpheme and a dictionary of sign-language words are separately provided; 
Fig. 22A shows a format of a dictionary 214 of sign-language word; 
Fig. 22B shows a format of a dictionary 212 of morpheme pattern; 

Fig. 23 is a block diagram showing the configuration of a detection part 211 of morpheme; 
Fig. 24 is a block diagram showing the configuration of a detection part 21 3 of word; 

Fig. 25 is a block diagram showing a system which utilizes an input apparatus according to a fourth embodiment 
for operating a computer; 

Fig. 26 is a block diagram showing the configuration of a recognition part 253 of gesture; 

Fig. 27 is a table showing a format of a gesture pattern stored in a dictionary 254 of gesture pattern; 

Figs. 28 A, 28 B show an example of a command which does not have additional information; 

Figs. 29A, 29B show an example of a command which has additional information; and 

Fig. 30 is a block diagram showing the configuration of a detection part 262 of gesture. 

[0020] The present invention will hereinafter be described by way of example with reference to Figs. 1-13 in con- 
nection with an embodiment in which the invention is applied to a continuous sign-language recognition apparatus. 
[0021 ] A block diagram of the whole configuration of a continuous sign-language recognition system is shown in Fig. 
2. Reference numeral 21 designates a pair of data gloves (Data Glove is a registered trade mark of VPL Research 
Lab., U.S.A.) for inputting a sign-language to the system. A sign-language input operator wears the data gloves 21 on 
his hands and "talks" a sign-language, with the result that the sign-language is inputted to the system. Reference 
numeral 22 designates an interface for converting an inputted sign-language word into a time-series continuous pattern 
of n-th order vector. With this interface 22, a sign-language pattern inputted by the data glove 21 is converted into time- 
series continuous sign-language patterns of n-th order vector which represent the position of hand, direction of palm, 
and figure of hand, respectively. For example, a sign-language word "people" as shown in Fig. 4A is converted into 
time-series patterns as shown in Fig. 4B. In Fig. 4B, patterns p1 - p3 are data representing the position of hand; d1 - 
d3 the direction of palm; and f1a - f5b the figure of hand, respectively. It should be noted that Fig. 4B only shows data 
on the right hand. Reference numeral 23 designates a dictionary of template pattern of sign-language for storing tem- 
plate patterns of sign-language sign-language word for the recognition of inputted patterns. Reference numeral 24 
designates a recognition part of sign-language word for matching an inputted sign-language pattern with template 
patterns of sign-language word to recognize a sign-language word expressed in the inputted sign-language pattern. 
Reference numeral 25 designates an output part for outputting candidates of the sign-language word recognized by 
the recognition part 24. 

[0022] Next, a format of the template pattern of sign-language word registered in the dictionary 23 of template pattern 
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of sign-language will be explained in detail with reference to Fig. 3. The template pattern of sign-language word consists 
of a word name 31 corresponding to a sign-language word pattern associated therewith, a number of morphemes 32 
constituting the pattern, and component patterns 33 of the respective morphemes. The morpheme is herein defined 
as a minimal unit having a meaning in patterns constituting a sign-language word pattern. Stated another way, the 
morpheme corresponds to "a word of the sign-language". However, since the morpheme does not correspond on one- 
to-one basis to the word in the natural language, it is assumed that a series of morphemes constitute a word. Further, 
a morpheme consists of three kinds of component patterns which are "position of hand" 34, "direction of palm" 35 and 
"figure of hand" 36, where each of the component patterns includes right hand data 37a and left hand data 37b. Further, 
the kind of pattern 38a and pattern data 38b are stored for each hand. The kind of pattern 3Ba represents whether an 
associated component pattern is a static pattern or a dynamic pattern. For a static pattern, an n-th order vector repre- 
sentative of the component pattern is stored as the pattern data 38b, while for a dynamic pattern, a time-series pattern 
of n-th order vector is stored. For example, a sign-language word shown in Fig. 4A may be expressed by patterns and 
data as shown in Fig. 4C. Thus, the patterns in the format as illustrated are stored for each sign-language word in the 
dictionary 23 of template pattern of sign-language. The break-down of a sign-language word pattern into morphemes 
and the registration of the morphemes in the dictionary may be readily achieved by manually extracting patterns rep- 
resenting a sign-language word inputted by the data gloves. However, since the morpheme is not clearly defined in 
the sign-language, how to break down an inputted sign-language word into morphemes is dependent on the determi- 
nation of the registration operator. 

[0023] In order to improve the recognition accuracy, additional parameters such as changing speed, direction of 
change, acceleration, and so on may be added to the above-mentioned parameters representing the position of hand, 
direction of palm and figure of hand. Also, each parameter constituting a pattern vector may be classified as being 
static or dynamic for registering the sign-language word. Further, each component pattern or each parameter of a 
pattern vector may be multiplied with a weighting coefficient indicative of an importance degree thereof with respect 
to a whole sign-language pattern associated therewith. The weighting can be utilized particularly for determining wheth- 
er a sign-language word is expressed by one hand or both hands. 

[0024] Next, the operation of the sign-language word recognition apparatus 2 will be explained in detail with reference 
to Figs. 1 , 5, 7, 8, 9 and 1 3. Fig. 1 is a block diagram showing the sign-language word recognition apparatus according 
to the present invention; Fig. 5 is a block diagram showing a matching part of pattern for matching an inputted sign- 
language pattern with the template patterns of sign-language word; Fig. 7 is a block diagram showing a detection part 
of pattern for detecting that a matched pattern is a static pattern; Fig. 8 is a block diagram showing a detection part of 
pattern for detecting that a matched pattern is a dynamic pattern; Fig. 9 is a block diagram showing a detection part 
of sign-language word for detecting a candidate of sign-language word from the pattern detection result; and Fig. 13 
is a block diagram showing a detection part of simultaneous gesture for detecting a simultaneous gesture in sign- 
language expressions. 

[0025] A matching part 11a of position pattern, a matching part 11b of direction pattern, and a matching part 11c of 
figure pattern respectively receive a position pattern d21a, a direction pattern d21b and a figure pattern d21c of a sign- 
language pattern d21 inputted from the interface 22, and match the respective received patterns of the inputted sign- 
language pattern with a position pattern d22a, a direction pattern d22b and a figure pattern d22c of each template 
pattern d22 of sign-language word stored in the dictionary 23 of template pattern of sign-language, respectively. In this 
event, for all morphemes constituting a word to be matched, each component pattern of the inputted sign-language 
pattern is matched with each component pattern of the template pattern of sign-language word. Also, the matching of 
the respective component patterns is carried out individually for right hand patterns and left hand patterns, respectively. 
[0026] The operations of the matching parts 11a, 11b, 11c of pattern will be explained in detail with reference to Fig. 
5. A matching process is executed respectively for patterns "position of hand", "direction of palm" and "figure of hand" 
by one and the same unit. First, each of selection parts 51 and 52 determines the kind of each of patterns contained 
in the template pattern d22 of sign-language word and selects a continuous DP matching part 53 or a calculation part 
54 of distance between vectors, to which the inputted sign-language pattern d21 and the template pattern d22 of sign- 
language word are to be inputted. Specifically, when the kind of pattern indicates static, the inputted sign-language 
pattern d21 and the template pattern d22 of sign-language word are supplied to the calculation part 54 of distance 
between vectors. In the calculation part 54 of distance between vectors, a calculation is made to the distance between 
an n-th order vector of the inputted sign-language pattern at each time of time-series data and an n-th order vector of 
the template pattern of sign-language word. This calculation of the distance employs, for example, a weighted Euclid 
distance as follows: 
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where 



d : distance between the pattern vectors; 
n : number of order of pattern vector; 
Wj : weight for the i-th order; 

Vj : i-th order data of inputted sign-language pattern; 

Pi : i-th order data of template pattern of sign-language word. 

[0027] Other than the Euclid distance, an equation commonly used in the pattern recognition such as a Mahaianobis 
distance, correlation between individuals, and so on may be used as the equation for the distance calculation. Alter- 
natively, a pattern classification and recognition technique such as vector quantization may also be used. This matching 
is executed at each time to obtain a time-series pattern of the distance as a curve d63 of Fig. 6. When the kind of 
pattern indicates dynamic, the inputted sign-language pattern d21 and the template pattern d22 of sign-language word 
are supplied to the continuous DP matching part 53. In the continuous DP matching part 53, the continuous DP matching 
is executed between the inputted sign-language pattern and the template pattern of sign-language word (see "Contin- 
uous Word Recognition using Continuous DP 0 , by Oka, the Speech Study G roup of Acoustical Society of Japan, S78-20, 
pp. 145 to 152, 1978). The continuous DP matching is a method of continuously matching two time-series patterns 
with each other while permitting expansion/contraction of time, wherein an inputted sign-language pattern is matched 
with template patterns of sign-language word at each time, so as to detect a matching start time at which two patterns 
(time-series patterns of n-th order vector) match with each other most closely. Then, the matching start time and the 
distance between the inputted sign-language pattern and the template pattern of sign-language word at that time are 
outputted. As a result, the distance between the inputted sign-language pattern and the template pattern of sign- 
language word is obtained as time-series data as indicated by curves d61 , d62 in Fig. 6. 

[0028] The method of matching static and dynamic patterns with template patterns of sign-language word is not 
limited to the distance calculation and the DP matching method as described above, but may be any method as long 
as it provides information for pattern detection such as the distance between patterns, the certainty factor on the ex- 
istence of a pattern, or the like. For example, the matching method may employ a neural network which receives pattern 
vectors at each time and learns which morphemes the inputted pattern vectors correspond to, such that the output of 
the neural network representative of the learning result is used for the matching. 

[0029] As a result of the matching as described above, for the matched morpheme, time-series data of the distance 
between the inputted sign-language pattern and the template pattern of sign-language word is outputted for each 
component pattern, as indicated by curves d61, d62, d63 in Fig. 6. In the example of Fig. 6, it is assumed that the 
component patterns "position of hand" and "direction of palm" are dynamic patterns, while "figure of hand" is a static 
pattern. The following explanation will be given on this assumption. 

[0030] The matching result from the calculation part 54 of distance between vectors is inputted to the detection part 
56 of static pattern, while the matching result from the continuous DP matching part 53 is inputted to the detection part 
55 of dynamic pattern. The operation of the detection part 56 of static pattern will be explained in detail with reference 
to Fig. 7. First, a detection part 71 of point crossing threshold detects a time at which an inputted distance d51 changes 
from a value equal to or more than a threshold to a value less than the threshold and a time at which the distance d51 
changes from a value less than the threshold to a value equal to or more than the threshold. A memory part 73 of 
changing time stores the time in response to a change signal d71 , generated from the detection part 71 of point crossing 
threshold, indicative of a change from a value equal to or more than the threshold to a value less than the threshold. 
Next, an output part 72 of information of static pattern, upon receiving a detection signal d72 indicative of a change 
from a value less than the threshold to a value equal to or more than the threshold, calculates a time length, during 
which a value less than the threshold remains, from the time d73 stored in the memory part 73 of changing time, and 
outputs the time d73 and the calculated time length. It will be understood that the detection part 56 of static pattern 
detects a portion from t31 to t32 of the curve d63 in Fig. 6. 

[0031] Next, the operation of the detection part 55 of dynamic pattern will be explained with reference to Fig. 8. A 
detection part 82 of minimum value detects a time at which the distance presents a minimum value from an inputted 
time-series pattern d52 of the distance. An output part 81 of information of dynamic pattern, upon receiving a minimum 
value detection signal d81 , calculates a time length of the pattern from a time at which the distance reaches the minimum 
value and a matching start time which gives the minimum distance. Then, these time values and time length are out- 
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putted. It will be understood that this detection part 55 of dynamic pattern detects the time points til , t12, 121 , t22 and 
the time lengths between t11 and t12 and between t21 and t22. 

[0032] The pattern matching parts 53, 54 and the pattern detection parts 55, 56 provide the result of matching an 
inputted sign-language pattern with morpheme patterns registered in the dictionary 23 of template pattern of sign- 
language for each of the component patterns (-position of hand", "direction of palm", and "figure of hand"). If the detected 
positions (information on pattern detected time and time length) of the respective component patterns overlap with 
each other, it can be identified that the input pattern corresponds to a matched morpheme. Further, if the detected 
positions of a sequence of detected morphemes fall under a predetermined tolerable range, the inputted pattern can 
be identified to be the sign-language word which has been matched therewith. This process is executed in the detection 
part 12 of sign-language word. The operation of the detection part 12 of sign-language word will be explained in detail 
with reference to Fig. 9. 

[0033] When the pattern detected positions dl 1 a, d1 1 b, d11 c of the component patterns are inputted to the detection 
part 1 2 of sign-language, a calculation part 91 of overlap first calculates the rate of overlap and the overlapping position 
for the detected positions of the three component patterns. The overlap of detected positions of the component patterns 
is calculated for all morphemes constituting a word to be detected. The rate of overlap is expressed by using a relative 
criterion based on the ratio of the length of the overlapping portion of the three component patterns to the overall length 
of the component patterns. For this purpose, the following equation may be applied by way of example: 

o = ™Ht12.t22,t32) - max(U 1J21J31) 

n\ax(t12,t22,t32) -m\n(t11,t21J31) (2) 

where 



o : rate of overlap of three kinds of component patterns; 

25 t1 1 , t1 2 : starting point and end point of pattern "position of hand"; 

t21 , t22 : starting point and end point of pattern "direction of palm"; 

t31 , t32 : starting point and end point of pattern "figure of hand". 
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With the three component patterns assumed in the relation as shown in Fig. 10, the rate of overlap is expressed by 
the ratio of the length from t101 to t102 to the length from t100 to t103. As another equation for calculating the rate of 
overlap, it is also possible to use an average of ratios of an overlapping portion of the three component patterns to the 
lengths of the respective component patterns. Alternatively, if the rate of overlap is likewise calculated not only for the 
case where all of the three components overlap with each other but also for the case where only two of them overlap, 
the recognition accuracy can be improved for a word which may often include a component pattern that fluctuates too 
largely to be detected. In this case, additional information should be added to indicate that only two component patterns 
are overlapping, or the equation for calculating the rate of overlap should include a penalty for taking account of the 
fact that overlapping component patterns are only two. When a morpheme consists of the three component patterns 
which are all static, the respective component patterns may occur at different timings, whereby no overlap is possibly 
detected in the three component patterns. It is therefore assumed only for such a case that the component patterns in 
a certain time range are regarded as being overlapping. More specifically, taking an example of component patterns 
placed in a relation illustrated in Fig. 11 A, since a gap till between the "position of hand" pattern and the "direction 
of palm" pattern is smaller than a preset threshold, these patterns are regarded to be overlapping. Conversely, in an 
example shown in Fig. 11B, since a gap t1 12 between the "position of hand" pattern and the "direction of palm" pattern 
is larger than the threshold, no overlap is determined to exist in these component patterns. When an overlap is recog- 
nized in this manner, the rate of overlap is always set to 1 .0. Alternatively, a different rate of overlap may be separately 
defined corresponding to a distance by which the two component pattern, regarded to be overlapping, are spaced, and 
used in place of the ordinary rate of overlap. 

[0034] The rate of overlap d91 of the component patterns calculated in the calculation part 91 of overlap is inputted 
to a decision part 93 of overlap, wherein if the rate of overlap is more than a threshold, it is determined that a matched 
morpheme has been detected from the inputted sign-language pattern, thus outputting a detection signal d92 from the 
decision part 93 to drive a comparison part 92 of position of overlap. The comparison part 92 of position of overlap 
determines whether or not a positional relation between one of morphemes constituting a word to be matched and a 
morpheme detected previously is within a tolerance value. The position of the previously detected morpheme has been 
stored in a memory part 95. This is a process required to a word to be detected which is constituted of a plurality of 
morphemes. With the first morpheme constituting a word, that is, if no information on morphemes constituting a word 
to be detected is stored in the memory part 95, the comparison part 92 of position of overlap unconditionally outputs 
current morpheme information. A decision part 94 of word compares information d95 in the memory part 95 and infor- 
mation d23 in the dictionary 23 of template pattern of sign-language with information d94 on the detected morphemes, 
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to determine whether a word has been detected. If it is determined that a word has been detected, the word and a 
detected position d96 are outputted. If a detected morpheme is not the last one constituting the word, information on 
that morpheme is stored in the memory part 95. Since ali morphemes constituting a word are in parallel detected in 
the detection of a word, a plurality of candidates of the word may be generated. For this reason, the memory part 95 
is preferably structured to permit a plurality of candidates of a word to be stored therein. 

[0035] Finally, a detection part 96 of simultaneous gesture examines whether a simultaneous gesture exists. The 
simultaneous gesture is an expression method peculiar to the sign-language and is such one that simultaneously 
expresses two sign-language words representing two actions. For example, a sign-language representing "to think 
while walking" is performed as shown in Fig. 12. The operation of the detection part 96 will be explained in detail with 
reference to Fig. 13. For the purpose of recognizing a simultaneous gesture, patterns expressed by one hand have 
previously been stored in the dictionary 23 of template pattern of sign-language for sign-language words which may 
possibly be combined to represent a simultaneous gesture. First, a selection part 131 of word selects words which 
may be used in a simultaneous gesture from candidates of words d96 sent from the decision part 94 of word by referring 
to the dictionary 23 of template pattern of sign -language. For a sign-language pattern which may express a simulta- 
neous gesture, candidates of word therefor are delivered to a calculation part 132 of overlap. Also, sign-language 
words normally expressed by one hand, which may be used for expressing a simultaneous gesture, are thought to be 
used for a single expression as well as a simultaneous gesture. For this reason, these sign-language words are out- 
putted as they are as candidates of word d24 as well as delivered to the calculation part 1 32 of overlap. Sign-language 
words which will not be used for a simultaneous gesture are outputted as they are as candidates of word d24. A sign- 
language word d131 which may be used for a simultaneous gesture is next supplied to the calculation part 132 of 
overlap for calculating the rate of overlap. In this case, words subjected to the calculation of the rate of overlap are 
limited to those including component patterns which do not overlap with each other. For example, if one sign-language 
word is expressed by the left hand, the other must be expressed by the right hand. This determination is made by 
referring to the contents d23 in the dictionary 23 of template pattern of sign-language. 

[0036] Calculation of the rate of overlap between words may be implemented by a similar method to the above 
described calculation of the rate of overlap for the component patterns. A calculated rate of overlap d132 is used in a 
decision part 133 to determine whether or not a simultaneous gesture is recognized. This determination may also be 
made whether or not the rate of overlap is equal to or more than a threshold, similarly to the aforementioned determi- 
nation performed by the decision part 94 of word. For words which are determined to represent a simultaneous gesture 
in the decision part 1 33 of overlap, a combination of two words are outputted as a candidate of word d24. 
[0037] In the above described embodiment, all parameters associated with the component patterns stored in the 
dictionary 23 of template pattern of sign-language are used to recognize sign-language words during the recognition 
process. The sign-language, however, may include parameters which are expressed by similar motion, figure and 
direction of hand and always indicate the same patterns, whichever context they are used in, and other parameters 
whose patterns change depending on a context to be expressed. In the latter case, even the meaning of certain sign- 
language patterns may change. For this reason, pattern data used in the dictionary 23 of template pattern of sign- 
language may be classified into a situation dependent parameter group and a situation independent parameter group, 
such that situation independent parameters are used in the recognition to detect words, while situation dependent 
parameters, which are thought to be necessary in subsequent processes, may be extracted after the detection of sign- 
language words. In this case, the dictionary 23 of template pattern of sign-language may take a format as shown in 
Fig. 14. Specifically, information on a pattern associated with each hand consists of kind of pattern 141, pattern data 
142 and additional information 143. The kind of pattern 141 and pattern data 142 are similar to those in Fig. 3. The 
additional information 143 describes which information is extracted from an associated pattern when a component 
pattern thereof is context dependent. Also, the detection part 15 of sign-language word may be modified as shown in 
Fig. 15 in order to process the additional information. A calculation part 151 of additional information calculates nec- 
essary information from a context dependent component pattern in accordance with the contents of the additional 
information registered in the dictionary 23 of template pattern of sign-language. 

(Embodiment 2) 

[0038] An embodiment which realizes the sign-language recognition apparatus 24 of the first embodiment by software 
will be described below with reference to Figs. 16-20. 

[0039] Fig. 16 is a flow chart showing an entire flow of a sign-language word recognition process; Fig. 17 is a flow 
chart showing a procedure of matching an input sign-language pattern with template patterns of sign-language word; 
Fig. 18 is a flow chart showing a procedure of detecting three component patterns, i.e., "position of hand", "direction 
of palm", and "figure of hand" from a matching result; Fig. 19 is a flow chart showing a procedure of a word detection 
process for detecting candidates of a sign-language word from the matching result; and Fig. 20 is a flowchart showing 
a procedure of a sign-language detection process for recognizing a simultaneous gesture from detected words. 
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[0040] Referring first to Fig. 16, at step 161 , a data for one unit time portion of a sign-language pattern inputted from 
the interface 22 is received, and an examination is made as to whether the currently inputted data is the last one If 
the current data is the last, the sign-language word recognition process is terminated. If further data remain, the pro- 
cedure advances to pattern matching processes for the respective component patterns at steps 162a, 162b, 162c, 
wherein the inputted sign-language pattern is matched with template patterns of sign-language word stored in the 
dictionary 23 of template pattern of sign-language. The pattern matching is executed for each of the component patterns 
"position of hand", "direction of palm" and "figure of hand". 

[0041] Now, the procedure of the pattern matching processes 162a, 162b, 162c will be explained in detail with ref- 
erence to Fig. 17. While the pattern matching processes are executed for the patterns "position of hand", "direction of 
palm" and "figure of hand", respectively, they can be carried out by completely the same process procedure. In each 
of the pattern matching processes 1 62a, 1 62b, 1 62c, data on one unit time portion of the inputted sign-language pattern 
is matched with all sign-language words registered in the dictionary 23 of template pattern of sign-language. For this 
purpose, it is first examined at step 171 whether or not there remain template patterns of sign-language word in the 
dictionary 23 of template pattern of sign-language which have not been matched with the inputted component pattern. 
If there remains no template pattern of sign-language word in the dictionary 23 to be matched with the inputted com- 
ponent pattern, the pattern matching process is terminated. If one or more template patterns of sign-language word to 
be matched remain in the dictionary 23, the procedure advances to step 172, where an examination is made as to 
whether the kind of a template pattern of sign-language word to be matched is static or dynamic by referring to the 
dictionary 23 of template pattern of sign-language as to the kind of the template pattern of sign-language word to be 
matched. When the template pattern to be matched is static, the procedure advances to step 173, where the inputted 
data is matched with the template pattern of sign-language word as being a static pattern. On the contrary, when the 
template pattern to be matched is dynamic, the procedure advances to step 174, where the inputted data pattern is 
matched with the template pattern as being part of a dynamic pattern. For the matching methods applied to the static 
pattern and the dynamic pattern, this embodiment employs the same methods as those which are used in the calculation 
part 54 of distance between vectors and the continuous DP matching part 53 in Fig. 5. 

[0042] After the inputted sign-language pattern has been matched with template patterns of sign-language word, the 
respective component patterns are detected at a position pattern detection step 163a, a direction pattern detection 
step 163b, and a figure pattern detection step 1 63c. respectively. While these processes are executed respectively for 
the patterns "position of hand", "direction of palm", and "figure of hand", they can be implemented by completely the 
same process procedure. The pattern detection process will now be explained in detail with reference to Fig. 18. In 
Fig. 18, it is first determined at step 181 whether a component pattern is static or dynamic by referring to the kind of 
pattern in a template pattern of sign-language word to be detected. For detecting a static pattern, it is determined at 
step 182 whether the distance has changed from a value equal to or more than a threshold to a value less than the 
threshold. If the distance has changed from a value equal to or more than the threshold to a value less than the 
threshold, the changing time is stored at step 183. Then, a state variable indicating whether the distance at that time 
is equal to or more than the threshold or less than the threshold is updated at step 184. This state variable is utilized 
at step 1 82 and in a determination process at step 185, later described. In the pattern detection processes 1 63a, 163b, 
163c, the state of the distance one unit time before is required. However, since these processes should be executed 
for each inputted matching result for one unit time portion of the component patterns, this state variable need be set 
for these processes to be executed next time. After the state variable has been updated, the procedure exits the pattern 
detection process steps 1 63a, 1 63b, 1 63c. If the determination at step 1 82 indicates that the distance has not changed 
from a value equal to or more than the threshold to a value less than the threshold, it is determined at step 1 85 whether 
the distance has changed from a value less than the threshold to a value equal to or more than the threshold. If the 
time at which the change has occurred is detected, the procedure advances to step 186, where the stored time at 
which the distance has changed from the value equal to or more than the threshold to the value less than the threshold 
and the current time are defined to be pattern detected positions. Thereafter, the procedure advances to step 184, 
where the state variable is updated. If a change of the distance from a value less than the threshold to a value equal 
to or more than the threshold is not detected at step 185, the procedure advances to step 184. 
[0043] Turning back to step 181, for detecting a dynamic pattern, the procedure advances to step 187, where a 
minimum value of the distance is detected. When the minimum value is detected, the time at which the minimum value 
is detected is defined to be an end point of the pattern, a starting point of the matching in this case is defined to be a 
starting point of the pattern, and the range between the starting point and the end point is defined to be the detection 
result. Subsequently, at step 189, a state variable, indicating whether the changing direction of the distance immediately 
before the minimum value detected time is in the decreasing direction or in the increasing direction, is updated and 
the distance at the minimum value detected time is stored. These state variable and distance are utilized for detecting 
the minimum value at step 187. If the minimum value is not detected at step 187, the procedure advances to step 189. 
[0044] After the respective component patterns of morphemes constituting each word have been detected, the pro- 
cedure advances to a word candidate detection process at step 164 (Fig. 16), where a candidate of sign-language 
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word is detected from the matching results. The procedure of the word candidate-detection process 164 will be ex- 
plained in detail with reference to Fig. 1 9. The word candidate detection process 1 64 detects a candidate of word from 
time-series data of the distance at each time which is the matching result of the inputted sign-language pattern with 
each template pattern of sign-language word. For this operation, an examination is first made at step 191 of Fig. 19 

s whether or not template patterns of sign-language word still remain so that the input sign-language pattern is to be 
matched with them for the word detection process. If no template pattern of sign-language word remains for the word 
detection process, this procedure is terminated. On the contrary, if one or more template patterns of sign-language 
word remain, the procedure advances to step 1 92, where, new overlaps between "position of hand", 'direction of palm" 
and "figure of hand" are searched for newly detected components and previously detected components, and the rate 

io of overlaps are calculated therefor. Such overlapping component patterns are searched in all combinations of patterns 
which represent morphemes constituting a word. The rate of overlap may be calculated by a method similar to that 
used in the calculation part 91 of overlap in Fig. 9. 

[0045] After new overlapping component patterns are searched at step 192, it is examined at step 193 whether 
overlapping component patterns to be evaluated still remain or not. If such overlapping component patterns still remain, 

'5 the following processes are executed for the respective overlapping component pattern sets. First at step 1 94, the rate 
of overlap is examined whether it is equal to or more than a threshold. If so, it is next determined at step 1 95 whether 
a gap between a newly detected morpheme and a previously detected morpheme is equal to or more than a threshold. 
This process is required when a word to be detected is constituted of a plurality of morphemes. If a morpheme is the 
first one constituting a word, the procedure unconditionally advances to step 196. If the gap is less than the threshold, 

20 jt is determined at step 1 96 whether or not the morpheme is the last one to be detected for recognizing the word. If it 
is the last one, the word and a detected position is outputted as the recognition result at step 1 98. Further, it is examined 
at step 199 whether or not a simultaneous gesture is concerned. The process of step 199 will be explained in detail 
with reference to Fig. 20. First at step 201, it is examined whether a detected word can be used in a simultaneous 
gesture. If it is a word used In a simultaneous gesture, already detected words are searched for detecting a word which 

25 can be used in a simultaneous gesture at step 202. If such a word is detected, the rate of overlap of the detected word 
with the newly detected word at step 202 is calculated at step 203. Then, an examination is made at step 204 whether 
or not the calculated rate of overlap is equal to or more than a threshold. If so, these two words are outputted as a 
combined word representing a simultaneous gesture. If the rate of overlap is less than the threshold, the procedure 
returns to step 202. At step 202, if no word which can be used in a simultaneous gesture is found from the previously 

30 detected words, the process is terminated. Also, if a word is determined to be one which cannot be used in a simulta- 
neous gesture at step 201, the procedure is terminated. 

[0046] Turning back to Fig. 1 9, if the determination at step 1 96 shows that a morpheme to be detected for recognizing 
a word is not the last one, the morpheme itself, a detected time thereof, and a sign-language word to be detected are 
stored at step 197, followed by the procedure returning to step 193. If the determination at step 195 shows that the 

35 gap is equal to or more than the threshold, and if the determination at step 194 shows that the rate of overlap is less 
than the threshold, the procedure returns to step 1 93. At step 1 93, if no overlapping component patterns to be evaluated 
remain, the procedure returns to step 191, where the above processes are repeated for a next template pattern of 
sign-language word. Since all morphemes constituting a word are in parallel detected in the word candidate detection 
process 164, a plurality of candidates of word may be stored at step 197. For this reason, the system should be con- 

40 figured so as to be able to store plural sets of information on a detected morpheme and a next morpheme. 

(Embodiment 3) 



45 



[0047] Next, an embodiment of a continuous sign-language recognition apparatus 

which has a dictionary 214 of sign-language word and a dictionary 212 of morpheme pattern separated from 
each other so as to store sign-language patterns in the unit of morpheme, will be described with reference to Figs 21 

- 24. 

[0048] Fig. 21 is a block diagram showing the whole configuration of a system which uses separated dictionary of 
sign-language word and dictionary of morpheme pattern. First in a detection part 211 of morpheme, recognition of 

50 morpheme is executed for all morpheme patterns described in the dictionary 212 of morpheme pattern. Next, in a 
detection part 213 of sign-language word, a sign-language word is recognized from morphemes constituting each of 
words which are described in the dictionary 214 of sign-language word. Figs. 22A and 22B show formats of the dic- 
tionary 214 of sign-language word and the dictionary 212 of morpheme pattern, respectively. As shown in Fig. 22 A, 
names of morphemes constituting respective sign-language words only are described in the dictionary 214 of sign- 

55 language word, while actual sign-language patterns are stored in the dictionary 212 of morpheme pattern shown in 
Fig. 22B. 

[0049] Next, Fig. 23 shows a block diagram of the detection part 211 of morpheme which has a combined configu- 
ration of the matching part of pattern in Fig. 5 and part of the detection part of sign-language word in Fig. 9. The 
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operations of the respective parts and a flow of data are similar to those shown in Figs. 5 and 9. It should be noted, 
however, that the operation of a decision part 231 of overlap slightly differs from its correspondent in the aforementioned 
embodiment. Specifically, if the rate of overlap calculated in a calculation part 231 of overlap is equal to or more than 
a threshold, a morpheme is determined to be detected, and the detected morpheme is outputted. 
[0050] Fig. 24 shows a block diagram of the detection part 213 of sign-language word which comprises several 
constituent parts extracted from the detection part of sign-language word of Fig. 9. The operations of the respective 
constituents are completely the same as those in Fig. 9. 

[0051] By separating the dictionary 212 of morpheme pattern and the dictionary 214 of sign-language word in the 
forgoing manner, pattern data common to a plurality of sign-language words may be stored only once, so that the 
storage capacity required to the dictionaries can be reduced. Also, by separately executing the detection of morpheme 
and the detection of sign-language word, the recognition of a morpheme common to a plurality of sign-language words 
may be executed only once, which results in a faster recognition process. 
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(Embodiment 4) 

[0052] Next, an embodiment of the present invention will be described with reference to Figs. 25 - 30 for the case 
where the invention is applied to an input apparatus for inputting a command to a computer in a form of a gesture which 
has been preset in correspondence to the command. 

[0053] Fig. 25 is a block diagram showing a system for utilizing an input apparatus 

for operating a computer. A command represented by a gesture is inputted through a pair of data gloves 251 and 
an interface 252. With these devices, information on the gesture is converted to n-th order time-series sequential data 
d252. The converted data is inputted to a recognition part 253 of gesture which uses gesture patterns d254 described 
in a dictionary 254 of gesture pattern to recognize the gesture representing a command to be inputted to a computer 
255. A keyboard 256 is further provided for directly inputting commands to operate the computer 255. 
[0054] The recognition part 253 of gesture will be explained in detail with reference to Fig 26. In the recognition part 
253 of gesture, patterns 'position of hand" d252a, "direction of palm" d252b, and "figure of hand" d252c representing 
an inputted gesture pattern d252 are supplied to a matching part 261a of position pattern, a matching part 261b of 
direction, and a matching part 261c of figure, respectively, where they are matched with position patterns d254a, di- 
rection patterns d254b, and figure patterns d254c stored in the dictionary 254 of gesture pattern. For the respective 
matching parts of the component patterns, the matching part shown in Fig. 5 may be used. It is required that the system 
enables the recognition part 253 of gesture to send, other than a command, additional information associated therewith 
to the computer 255 in order to realize flexible operations. Such additional information may be, for a command for 
instructing the computer 255 to execute a pointing operation, positional information indicative of a pointed location, or 
the like. For this purpose, a detection part 262 of gesture is required to be capable of simultaneously detecting a 
command as well as such additional information associated with the command. The registration in the dictionary 254 
of gesture pattern is done in the form of the dictionary 214 of sign-language pattern shown in Fig. 14, and additional 
information is also described in the dictionary 254 as to what information is detected from which component pattern. 
[0055] The format of gesture patterns described in the dictionary 254 of gesture pattern will be explained in detail 
with reference to Fig. 27. Described in the dictionary 254 of gesture pattern are a command name 271 , a number 272 
of gestures representing a command, and information 273 on respective actual gestures. As further information on the 
respective gestures, component data "figure of hand" 274, "direction of palm" 275, and "position of hand" 276 are also 
described in the dictionary 254 of gesture pattern. The respective component data is divided into data on the right hand 
and data on the left hand. For each hand data, kind of pattern 277 indicative of a static pattern or a dynamic pattern, 
and actual pattern data 278 corresponding to each pattern are described. Further, for detecting the additional informa- 
tion such as a pointed location, each hand data includes a description for specifying which data is calculated from 
gesture data. If any of the pattern kind description , the actual pattern data, and the additional information is not described 
for a particular hand data, the matching part is controlled so as not to refer to such absent data during the matching 
process or the detection of the additional information. Fig. 28 shows an exemplary description in the dictionary 254 of 
gesture pattern for defining a command which has no additional information. In this case, a command A is sent to the 
computer 255 by detecting a pattern representing a certain gesture at a particular location while maintaining particular 
figure of hand and direction of palm. Fig. 29 shows an exemplary description in the dictionary 254 of gesture pattern 
for defining a command which requires additional information. This command B is used when a particular location on 
a monitor of the computer is pointed by a finger pointing operation, or the like. As data for matching, only the component 
pattern "figure of hand" is stored. For the component patterns "position of hand" and "direction of palm", there are 
descriptions for instructing to calculate additional information. The data on the pattern "figure of hand", which represents 
a hand with an erected index finger, is an element determining that this is a finger pointing command. The addition 
information of the component patterns "position of hand" and "direction of palm" are utilized by the computer 255 as 
information representing the direction in which the index finger is pointing when the component pattern "figure of hand" 
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representing this command is detected. By describing a command in this format, the matching with flexible descriptions 
can be achieved even for commands which do not depend on changes in a particular component and for the case 
where a change in a particular pattern causes the meaning of a command to change. 

[0056] The operation of the detection part 262 of gesture will be explained in detail with reference to Fig. 30. First, 

5 in a calculation part 301 of overlap, an overlap of detected component patterns is found. In this event, since the three 
component patterns are not always sent depending on a command, an overlap is found for necessary component 
patterns in accordance with contents d254 described in the dictionary 254 of gesture pattern. With the command shown 
in Fig. 28, an overlap is found for all the three component patterns. Contrarily, with the command shown in Fig. 29, 
since detected data on the component pattern "figure of hand" only is sent, an overlap need not be found, so that the 

w detected data is outputted from the calculation part 301 as it is. A decision part 303 of overlap determines whether the 
found overlap amount d301 is equal to or more than a threshold. If so, a comparison part 302 of position of overlap is 
driven. The comparison part 302 of position of overlap determines, for a gesture representing a command to be de- 
tected, whether a temporal relation between a gesture d301 detected at that time and a gesture d303 detected before 
that time is within a tolerance value. In a memory part 305, information on gestures detected before when new gesture 

is js detected is stored. If no gesture information is stored in the memory part 305, detected gesture information is out- 
putted as it is from the comparison part 302. A decision part 304 of command compares the detected gesture d304 
and gesture information d305 stored in the memory part 305 with the contents d254 of the dictionary 254 of gesture 
pattern to determine whether a command is detected. If a command is detected, additional information is calculated 
in a calculation part 306 of additional information in accordance with a description in the dictionary 254 of gesture 

20 pattern. With the command shown in Fig. 29, "position of hand" and "direction of finger" are calculated from a part in 
the input pattern d252 corresponding to the command, and outputted from the calculation part 306. 
[0057] The detected command and additional information associated therewith are sent to the computer 255 wherein 
the sent command and additional information are received by an application such that an operation is performed by 
the computer 255 in accordance with the command. 

25 [0058] As described above, 

for creating a template pattern of sign-language word, component patterns constituting the template pattern are 
classified into a static pattern and a dynamic pattern, and the static pattern is expressed by a vector representing the 
pattern to reduce a storage capacity required for storing template patterns of sign-language word. Also, component 
patterns constituting a template pattern of sign-language word are classified into a static pattern and a dynamic pattern, 

30 and different recognition methods are utilized for the static and dynamic patterns, respectively, whereby a recognition 
time can be shortened as compared with a conventional recognition method which employs one and the same recog- 
nition method for all patterns. Further, by separately matching component patterns constituting an input sign-language 
pattern with component patterns constituting template patterns of sign-language word, flexible recognition may be 
carried out even for the case where a displacement of the respective component patterns causes a degradation of the 

35 recognition accuracy, and for the case where the recognition accuracy is reduced due to a largely fluctuating component 
pattern. Since static patterns as well as dynamic patterns are in parallel matched before determining which patterns 
are valid, erroneous determination as to whether a pattern is static or dynamic is reduced, with the result that the 
recognition accuracy of the sign-language word can be improved. 



1. A continuous sign-language recognition system comprising: 

means for converting a series of sign-language words into sign-language patterns which are time-series pat- 
terns of feature vectors and inputting said converted sign-language patterns; 
means for storing template patterns for sign-language words; 

pattern matching means for matching said input sign-language patterns with said template patterns of sign- 
language word, to recognize a sign-language word expressed in said input sign-language patterns; 
recognizing means responsive to said matching means for recognizing a sign-language word from the results 
of matching between the input sign-language pattern and the template patterns; and 
means for outputting the sign-language word recognized by said recognizing means; 
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characterized in that: 
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each of said template patterns for sign-language words includes a description for identifying whether each of 
gesture elements each constituting a sign-language word is a static pattern or a dynamic pattern, and a pattern 
of the gesture element expressed as a form at that is different depending upon whether the gesture element 
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is a static pattern or a dynamic pattern; 

said pattern matching means, referring to said template patterns in said storing means, selecting a relevant 
one of different matching schemes in accordance with whether each gesture element in the template patterns 
is static or dynamic and performing a matching of said input sign language patterns with the pattern of each 
gesture element using the selected matching scheme, independently from the other gesture elements; and 
said sign-language word recognizing means integrating recognition results of the gesture elements each con- 
stituting a sign-language word, based on a time relation with the recognition results. 

A continuous sign-language recognition system according to claim 1, wherein the gesture element patterns of 
sign-language word in the template patterns is expressed as a feature vector for the sign-language word having 
a static pattern, and as a time-series pattern of a feature vector for the sign-language word having a dynamic 
pattern. 

A continuous sign-language recognition system according to claim 1 , wherein each of said gesture element patterns 
is weighted with a weighting coefficient indicative of an importance degree thereof with respect, to the whole pattern, 
and pattern matching is performed using said weighting coefficient. 

A continuous sign-language recognition system according to claim 1 , wherein said sign-language word recognition 
means calculates an overlap of detection positions between gesture elements detected by the matching of the 
gesture element patterns, and determines that a sign-languages word corresponding to a template pattern matched 
with the input sign-language pattern is present, when the calculated overlap for inter-gesture elements exceeds a 
predetermined value. 

A continuous sign-language recognition system according to claim 4, wherein when all of gesture element patterns 
constituting a sign-language word are static patterns, said sign-language word recognition means determines that 
a sign-language word corresponding to a template pattern matched with the input sign-language pattern is present 
in response to a detection of the detected gesture element within a predetermined time range. 

A continuous sign-language recognition system according to claim 4, wherein said sign-language word recognition 
means detects concur rent operation in a sign language from detection results of a plurality of sign-language word 
patterns, based on an overlap thereof. 

A continuous sign-language recognition system according to claim 1, wherein said template patterns of sign lan- 
guage word are expressed by a variation in the gesture element pattern and appended information indicative of a 
semantic variation of the gesture element pattern, and said sign-language word recognition means detects the 
variation of an expression or format of the gesture element pattern, and calculates and extracts appended infor- 
mation described in the template pattern. 

A continuous sign-language recognition system comprising: 

means for converting a series of sign-language words into sign-language patterns which are time-series pat- 
terns of feature vectors and inputting said sign-language patterns; 
pattern storage means for storing template patterns of sign-language morphemes; 

word store means for storing sign-language words each expressed as a combination of sign-language mor- 
phemes constituting a sign-language word; 

pattern matching means for matching said sign-language patterns with said template patterns of sign-language 
morphemes, in order to recognize sign-language morphemes expressed in said sign-language patterns; 
morpheme recognition means responsive to said pattern matching means for recognizing sign-language mor- 
phemes from the results of matching between one of said sign-language patterns and the template patterns 
of sign-language morphemes; 

word recognition means connected to said morpheme recognition means for recognizing a sign-language 
word by matching the recognized sign-language morphemes with a combination of sign-language morphemes 
constituting a sign-language word; and 

means for outputting the sign-language word recognized by said word recognizing means; 
said template patterns stored in said sign-language morpheme template storing means including a description 
of identifying whether each of gesture elements constituting a sign-language morpheme is a static pattern or 
a dynamic pattern, and a pattern of the gesture element expressed as a format different depending upon 
whether the gesture element is a static pattern or a dynamic pattern; 
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said pattern matching means matching the input sign-language patterns with the sign-language morpheme 
template patterns, referring to said template patterns in said pattern storage means, selecting a relevant one 
of different matching schemes in accordance with whether each gesture element in the template patterns is 
static or dynamic, and performing a matching of said sign language patterns with the pattern of each gesture 
element using the selected matching scheme, independently from the other gesture elements; and 
said morpheme recognition means integrating recognition results of the gesture elements each constituting a 
sign-language morpheme, based on a time relation with the recognition results to recognize the sign-language 
morpheme. 

9. An input system comprising: 

sign-language-input means for converting a series of gestures into gesture patterns which are time-series 
patterns of feature vectors and inputting the converted gesture patterns; 
storing means for storing template patterns of gestures; 

matching means for matching the input gesture patterns with the template patterns of the gesture stored in 
said storing means, in order to recognize a gesture expressed in the gesture pattern; 

gesture recognizing means responsive to said matching means for recognizing the gesture expressed in the 
gesture pattern, from the results of matching between the input gesture patterns and the template patterns; and 
means for performing a processing corresponding to the recognized gesture; 

each of said gesture template patterns stored in said template pattern storing means including a description 
of identifying whether each of gesture elements constituting a gesture is a static pattern or a dynamic pattern, 
and a pattern of the gesture element expressed as a format different depending upon whether the each gesture 
element is a static pattern or a dynamic pattern, 

said matching means for matching the input gesture patterns with the gesture template patterns, referring to 
said template patterns in said storing means, selecting a relevant one of different matching schemes in ac- 
cordance with whether each of gesture elements in the template patterns is static or dynamic and performing 
a matching of said gesture patterns with the pattern of each gesture element using the selected matching 
scheme, independently from the other gesture elements; and 

said gesture recognizing means integrating recognition results of the gesture elements constituting a gesture, 
based on a time relation with the recognition results. 

10. An input system according to claim 9, wherein the template of the gesture is expressed as a feature vector for the 
gesture having a static pattern, and as a time-series pattern of a feature vector for the gesture having a dynamic 
pattern. 

11 . An input system according to claim 9, wherein said gesture recognition means calculates an overlap of recognition 
(detection) positions between gesture elements recognized (detected) by the matching of the gesture element 
patterns, and determines that a gesture corresponding to a template pattern matched with the input gesture pattern 
is present, when the calculated overlap for inter-gesture elements exceeds a predetermined value. 

12. An input system according to claim 11, wherein when all of gesture e elements constituting a gesture are static 
patterns, said gesture recognizing means determines that a gesture corresponding to a template pattern matched 
with the input gesture e pattern is present in response to a detection of the detected gesture e element within a 
predetermined time range. 

13. An input system according to claim 12, wherein the gesture template pattern is expressed by a pattern of gesture 
elements and an appended information indicative of data to be extracted from the gesture pattern; and 

said gesture recognizing means calculates and extract necessary data from the gesture pattern based on the 
recognition result of the gesture and the appended information in the template pattern. 



Patentanspruche 

1. Vorrichtung zur kontinuierlichen Erkennung von Gebardensprache, umfassend: 

eine Einrichtung zum Umwandeln einer Gebardensprache-Wortfolge in Gebardensprachmuster, die Zeitfol- 
gemuster von Eigenschaftsvektoren sind, und zur Eingabe der umgewandeften Gebardensprachmuster, 
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eine Einrichtung zum Speichern von Gebardensprachenwort-Referenzmustern, 

eine Musterzuordnungseinrichtung zum Zuordnen der eingegebenen Gebardensprachmuster zu den Gebar- 
densprachenwort-Referenzmustem, urn ein Gebardensprachenwort, das in einem eingegebenen Gebarden- 
sprachmuster ausgedruckt ist, zu erkennen, 

eine Erkennungseinrichtung, dieauf die Zuordnungseinrichtung reagiert, zum Erkennen eines Gebardenspra- 
chenwortes aus dem Ergebnis des Vergleichs zwischen dem eingegebenen Gebardensprachmuster und dem 
Referenzmuster, und 

eine Einrichtung zur Ausgabe des von der Erkennungseinrichtung erkannten Gebardensprachenwortes, 
dadurch gekennzeichnet: 

daB jedes der Referenzmuster fur Gebardensprachenworte eine Beschreibung zum Erkennen enthalt, ob 
jedes ein Gebardensprachenwort darsteilendes Gebardenelement ein statisches oder bewegtes Muster ist 
und ein Muster eines Gebardenelements enthalt, das in unterschiedlicher Form ausgedruckt wird, je nachdem, 
ob das Gebardenelement ein statisches oder ein bewegtes Muster ist, 

daB die Musterzuordnungseinrichtung auf die Referenzmuster in der Speichereinrichtung zuruckgreift, aus 
unterschiedlichen Zuordnungsprogrammen ein entsprechendes auswahlt, je nachdem, ob das Gebardenele- 
ment in den Referenzmustern statisch oder bewegt ist, und unabhangig von anderen Gebardenelementen 
einen Vergleich der eingegebenen Gebardensprachenmuster mit den Mustern jedes Gebardenelementes un- 
ter Verwendung des ausgewahlten Zuordnungsprogramms durchfuhrt, und 

die Gebardensprachenwort-Erkennungseinrichtung die Erkennungsergebnisse der jeweils ein Gebardenspra- 
chenwort darstellenden Gebardenelemente auf der G rundlage einer zeitlichen Beziehung zu den Erkennungs- 
ergebnissen integriert. ' 

Vorrichtung zur kontinuierlichen Erkennung von Gebardensprache nach Anspruch 1 , wobei die Gebardenelement- 
muster eines Gebardensprachenwortes in den Referenzmustern als Eigenschaftsvektor fur ein Gebardenspra- 
chenwort mit einem statischen Muster und als Zeitfolgemuster eines Eigenschaftsvektors fur ein Gebardenspra- 
chenwort mit dynamischem Muster ausgedruckt sind. 

Vorrichtung zum kontinuierlichen Erkennen von Gebardensprache nach Anspruch 1, wobei jedes Gebardenele- 
mentmuster mit einem Gewichtungskoeffizienten gewichtet wird, der den Wichtigkeitsgrad in bezug auf das ge- 
samte Muster anzeigt, und bei der die Musterzuordnung unter Verwendung des Gewichtungskoeffizienten durch- 
gef uhrt wird. 

Einrichtung zum kontinuierlichen Erkennen von Gebardensprache nach Anspruch 1, wobei die Gebardenspra- 
chenwort-Erkennungseinrichtung eine Uberschneidung von Erkennungspositionen rechnet, die beim Vergleich 
der Gebardenelementmuster erkannt wurden, und bestimmt, daB ein Gebardensprachenwort vorliegt, das einem 
Referenzmuster entspricht, welches mit dem eingegebenen Gebardensprachenmuster ubereinstimmt, wenn die 
berechnete Uberschneidung fur Zwischengebardenelemente einen vorbestimmten Wert Oberschreitet. 

Vorrichtung zum kontinuierlichen Erkennen von Gebardensprache nach Anspruch 4, bei der, wenn alle ein Ge- 
bardensprachenwort darstellenden Gebardenelementmuster statische Muster sind, die Einrichtung zum Erkennen 
von Gebardensprachenworten festlegt, daB ein Gebardensprachenwort, das zu einem Referenzmuster gehort, 
welches zum eingegebenen Gebardensprachenmuster paBt, vorliegt, als Antwort auf das Erkennen des erkannten 
Gebardenelements in einem vorbestimmten Zeitraum. 

Vorrichtung zum kontinuierlichen Erkennen von Gebardensprache nach Anspruch 4, bei der die Gebardenspra- 
chenwort-Erkennungseinrichtung konkurrierende Operationen aus den Erkennungsergebnissen mehrerer Gebar- 
densprachen-Wortmuster auf der Basis ihrer Uberschneidungen erkennt. 

Vorrichtung zum kontinuierlichen Erkennen von Gebardensprache nach Anspruch 1 , bei der die Referenzmuster 
der Gebardensprachenworte durch Variationen in den Gebardenelementmustern und anhangenden Informatio- 
nen, die semantische Variationen dieser Muster anzeigen, ausgedruckt sind, und die Gebardensprachenwort- 
Erkennungseinrichtung die Variation eines Ausdrucks oder eines Formats eines Gebardenelementmusters erkennt 
und die in den Referenzmustern beschriebenen angehangte Information berechnet und herauszieht. 



Vorrichtung zum kontinuierlichen Erkennen von Gebardensprache, umfassend: 
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eine Einrichtung zum Umwandeln einer Gebardensprachen-Wortserie in Gebardensprachenmuster, die Zeit- 
folgenmuster von Eigenschaftsvektoren sind, und zum Eingeben der Gebarden-sprachmuster, 
eine Musterspeichereinrichtung zum Speichern von Referenzmustern der Gebardensprachen-Morpheme, 
eine Wortspeichereinrichtung zum Speichern von Gebardensprachenworten, die jeweils ais eine ein Gebar- 
densprachenwort darstellende Kombination von Gebardensprachen-Morphemen ausgedruckt werden, 
eine Musterzuordnugseinrichtung zum Zuordnen der Gebardensprachenmuster zu den Referenzmustern der 
Gebardensprachen-Morpheme, um die Gebardensprachen-Morpheme zu erkennen, die in Gebardenspra- 
chenmustern ausgedruckt sind, 

eine auf die Mustervergleichseinrichtung ansprechende Morphemerkennungseinrichtung zum Erkennen von 
Gebardensprachen-Morphemen aus den Vergleichsergebnissen zwischen den Gebardensprachenmustem 
und den Referenzmustern der Gebardensprachen-Morpheme, 

eine mit der Morphemerkennungseinrichtung verbundene Worterkennungseinrichtung zum Erkennen eines 
Gebardensprachenwortes durch Vergleich der erkannten Gebardensprachenmorpheme mit einer Gebarden- 
sprachen-Morphemkombination, die ein Gebardensprachenwort darstellt, und 

eine Einrichtung zur Ausgabe des von der Worterkennungseinrichtung erkannten Gebardensprachenwortes, 
die im Gebardensprache-Morphemreferenzspeicher gespeicherten Referenzmuster, die eine Erkennungsbe- 
schreibung enthalten, ob ein ein Gebardensprachen-Morphem darstellendes Gebardenelement ein statisches 
Oder dynamisches Muster hat (ist) und das Muster eines Gebardenelements in einem unterschiediichen For- 
mat ausgedruckt wird, je nachdem, ob das Gebardenelement ein statisches oder ein dynamisches Muster ist, 
die Musterzuordnungseinrichtung, die die eingegebenen Gebardensprachenmuster den Gebardensprachen- 
morphem-Referenzmustern zuordnet, indem sie auf die Referenzmuster in den Musterspeichern zuruckgrei- 
fen, aus verschiedenen Vergleichsmustern ein entsprechendes auswahlen, je nachdem, ob das Gebarden- 
element in den Referenzmustern statisch oder dynamisch ist, und einen Vergleich des Gebardensprachmu- 
sters mit dem Muster jedes Gebardenelements unter Verwendung des ausgewahlten Zuordnungsprogramms 
unabhangig von den anderen Gebardenelementen durchfuhren, und 

die Morphemerkennungseinrichtung, die die Erkennungsergebnisse der jeweils ein Gebardensprachmorphem 
darstellenden Gebardenelemente auf einer Zeitbeziehung zwischen den Erkennungsergebnissen basierend 
integriert, um das Gebardensprachen-Morphem zu erkennen. 

9. Eine Eingabevorrichtung, umfassend: 

eine Gebardensprache-Eingabeeinrichtung zum Umwandeln einer Serie von Gebarden in Gebardenmuster, 
die Zeitfolgemuster von Eigenschaftsvektoren sind und zum Eingeben der umgewandelten Gebardenmuster, 
eine Speichereinrichtung zum Speichern von Referenzmustern von Gebarden, 

eine Zuordnungseinrichtung zum Zuordnen der eingegebenen Gebardenmuster zu den Referenzmustern der 
in der Speichereinrichtung gespeicherten Gebarden, um eine in einem Gebardenmuster ausgedruckte Ge- 
barde zu erkennen, 

Gebardenerkennungseinrichtung. die auf die Zuordnungseinrichtung reagiert, um aus den Ergebnissen des 
Vergleichs der eingegebenen Gebardenmuster mit den Referenzmustern die in dem Gebardenmuster ausge- 
druckte Gebarde zu erkennen, und 

eine Einrichtung zum Durchfuhren eines zur erkannten Gebarde gehorenden Vorgangs, 
im Referenzmusterspeicher gespeicherte Gebardenreferenzmuster, deren jedes eine Erkennungsbeschrei- 
bung enthalt, ob das eine Gebarde darstellende Gebardenelement ein statisches oder dynamisches Muster 
ist und ein Muster des Gebardenelementes in einem unterschiediichen Format ausdruckt, je nachdem, ob das 
jeweilige Gebardenelement ein statisches oder dynamisches Muster ist, 

die Zuordnungseinrichtung zum Zuordnen der eingegebenen Gebardenmuster zu den Gebardenreferenzmu- 
stern, die auf die Referenzmuster in der Speichereinrichtung bezug nimmt, ein relevantes verschiedener ver- 
gleichsschemata aussucht, je nachdem, ob das jeweilige Gebardenelement in den Referenzmustern statisch 
oder dynamisch ist, und unter Verwendung des ausgewahlten Vergleichsschemas, unabhangig von den an- 
deren Gebardenelementen, einen Vergleich der Gebardenmuster mit den Mustern jedes Gebardenelementes 
ausfuhrt, und 

die Gebardenerkennungseinrichtung, die die Erkennungsergebnisse der eine Gebarde darstellenden Gebar- 
denelemente auf der Basis einer Zeitrelation zu den Erkennungsergebnissen integriert. 

10. Eingabevorrichtung nach Anspruch 9, bei der die Referenzgebarde als Eigenschaftsvektor fur eine Gebarde mit 
statischem Muster und als Zeitfolgemuster eines Eigenschaftsvektors fur eine Gebarde mit dynamischem Muster 
ausgedruckt wird. 
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11. Eingabevorrichtung nach Anspruch 9, bei der die Gebar-denerkennungseinrichtung eine Uberschneidung von Er- 
kennungspositionen zwischen Gebardenelementen, die durch den Vergleich von Gebardenelementrnustern er- 
kannt wurden, berechnet und festlegt, daQ eine Gebarde vorliegt, die zu einem Referenzmuster gehort, welches 
dern eingegebenen Gebardenmuster zugeordnet ist, wenn die berechnete Uberschneidung fur Zwischen-Gebar- 
denelemente einen vorbestimmten Wert uberschreitet. 

12. Eingabevorrichtung nach Anspruch 11, bei dem, wenn alle eine Gebarde bildenden Gebardenelemente statische 
Muster sind, die Gebardenerkennungseinrichtung als Reaktion auf das Erkennen eines erkannten Gebardenele- 
mentes in einem vorbestimmten Zeitraum festlegt, daG eine Gebarde vorliegt, die zu einem Referenzmuster gehort, 
welches der eingegebenen Gebarde zugeordnet worden ist. 

13. Eingabevorrichtung nach Anspruch 12, bei der das Gebardenreferenzmuster durch eih Muster von Gebardenele- 
menten und eine angehangte Information, die aus dem Gebardenmuster herauszulesende Daten anzeigt, ausge- 
druckt wird, und 



die Gebardenerkennungseinrichtung notwendige Daten, die auf dem Erkennungsergebnis der Gebarde und 
der angehangten Information im Referenzmuster basieren, berechnet und herausliest. 



Revendications 

1. Systeme de reconnaissance continue de langage des signes, comportant : 

des moyens pour convertir une serie de mots de langage des signes en des motifs de langage des signes qui 
sont des motifs serie de vecteurs caracteristiques et entrer lesdits motifs de langage des signes convertis, 
des moyens pour memoriser des motifs types pour des mots de langage des signes, 

des moyens de confrontation de motifs pour confronter lesdits motifs de langage des signes qui ont ete entres 
auxdits motifs types de mot de langage des signes, afin de reconnaTtre un mot de langage des signes exprime 
par lesdits motifs de langage des signes qui ont ete entres, 

des moyens de reconnaissance sensibles auxdits moyens de confrontation pour reconnaTtre un mot de lan- 
gage des signes a partir des resultats de la confrontation effectuee entre le motif de langage des signes qui 
a ete entre et les motifs types, et 

des moyens pour delivrer en sortie le mot de langage des signes qui a ete reconnu par lesdits moyens de 
reconnaissance, 



caracterise en ce que : 

chacun desdits motifs types pour des mots de langage des signes comporte une description qui permet d'iden- 
tifier si chaque element gestuel constituant un mot de langage des signes est un motif statique ou un motif 
dynamique, et un motif de I'element gestuel exprime sous une forme qui differe selon que I'element gestuel 
est un motif statique ou un motif dynamique, 

lesdits moyens de confrontation de motifs, en se reportant auxdits motifs types dans lesdits moyens de me- 
morisation, selectionnant un schema pertinent parmi differents schemas de confrontation selon que chaque 
element gestuel dans les motifs types, est statique ou dynamique et conf rontant lesdits motifs de langage des 
signes qui ont et6 entres au motif de chaque element gestuel en utilisant le schema de confrontation selec- 
tionne, inddpendamment des autres elements gestuels, et 

lesdits moyens de reconnaissance de mot de langage des signes integrant les resultats de la reconnaissance 
des elements gestuels constituant chacun un mot de langage des signes, sur la base d'une relation temporelle 
avec les resultats de la reconnaissance. 



2. Systeme de reconnaissance continue de langage des signes selon la revendication 1, dans lequel les motifs 
d'element gestuel de mot de langage des signes, dans les motifs types, sont exprimes sous la forme d'un vecteur 
caracteristique dans le cas ou ie mot de langage des signes possede un motif statique, et sous la forme d'un motif 
serie d'un vecteur caracteristique lorsque le mot de langage des signes possede un motif dynamique. 

3. Systeme de reconnaissance continue de langage des signes selon la revendication 1 , dans lequel chacun desdits 
motifs d'un element gestuel est pondere par un coefficient de ponderation indicatif de son degre d'importance par 
rapport a I'ensemble du motif, et dans lequel la confrontation des motifs est effectuee en utilisant ledit coefficient 
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de ponderation. 

Systeme de reconnaissance continue de Ian gage des signes selon la revendication 1 , dans lequel lesdits moyens 
de reconnaissance de mot de langage des signes calculent un chevauchement de positions de detection entre 
elements gestuels detectes par la confrontation des motifs d'elements gestuels, et determinent qu'un mot de lan- 
gage des signes correspondant a un motif type confronte au motif de langage des signes qui a 6t6 entre est present 
lorsque le chevauchement calcule entre elements gestuels excede une valeur pred^terminee. 

Systeme de reconnaissance continue de langage des signes selon la revendication 4, dans lequel lorsque tous 
les motifs d'elements gestuels qui constituent un mot de langage des signes sont des motifs statiques, lesdits 
moyens de reconnaissance de mot de langage des signes determinent qu'un mot de langage des signes corres- 
pondant a un motif type confronte au motif de langage des signes qui a ete entr6 est present en reponse a la 
detection de I'element gestuel detecte au cours d'un intervalle de temps predetermined 

Systeme de reconnaissance continue de langage des signes selon la revendication 4, dans lequel lesdits moyens 
de reconnaissance de mot de langage des signes detectent une operation concurrente dans un langage des signes 
a partir des resultats de detection d'une pluralite de motifs de mot de langage des signes, sur la base de leur 
chevauchement. 

Systeme de reconnaissance continue de langage des signes selon la revendication 1 , dans lequel lesdits motifs 
types de mot de langage des signes sont exprimes par une variation dans le motif d'element gestuel et par des 
informations annexees indicatives d'une variation semantique du motif d'element gestuel, et lesdits moyens de 
reconnaissance de mot de langage des signes detectent la variation d'une expression ou d'un format du motif 
d'element gestuel, et calculent et extraient les informations annexees decrites dans le motif type. 

Systeme de reconnaissance continue de langage des signes comportant : 

des moyens pour convertir une serie de mots de langage des signes en des motifs de langage des signes qui 

sont des motifs serie de vecteurs caracteristiques et entrer lesdits motifs de langage des signes, 

des moyens de memorisation de motifs pour memoriser des motifs types de morphemes de langage des 

signes, 

des moyens de memorisation de mot pour memoriser des mots de langage des signes exprimes chacun par 
une combinaison de morphemes de langage des signes constituant un mot de langage des signes, 
des moyens de confrontation de motifs pour confronter lesdits motifs de langage des signes auxdits motifs 
types de morphemes de langage des signes, afin de reconnaTtre les morphemes de langage des signes ex- 
primes dans lesdits motifs de langage des signes, 

des moyens de reconnaissance de morpheme sensibles auxdits moyens de confrontation de motifs pour 
reconnaUre les morphemes de langage des signes a partir des resultats de la confrontation effectu£e entre 
t'un desdits motifs de langage des signes et les motifs types de morphemes de langage des signes, 
des moyens de reconnaissance de mot relies auxdits moyens de reconnaissance de morpheme pour recon- 
naUre un mot de langage des signes en conf rontant les morphemes de langage des signes qui ont ete reconnus 
avec une combinaison de morphemes de langage des signes constituant un mot de langage des signes, et 
des moyens pour delivrer en sortie le mot de langage des signes reconnu par lesdits moyens de reconnais- 
sance de mot, 

lesdits motifs types memorises dans lesdits moyens de memorisation de motifs type de morphemes de langage 
des signes comportant une description qui permet d'identifier si chacun des elements gestuels constituant un 
morpheme de langage des signes est un motif statique ou un motif dynamique, et un motif d'element gestuel 
exprime sous une forme qui differe selon que I'element gestuel est un motif statique ou un motif dynamique, 
lesdits moyens de confrontation de motifs, confrontant les motifs de langage des signes qui ont 6te entres 
aux motifs types de morphemes de langage des signes, en se reportant auxdits motifs types dans lesdits 
moyens de memorisation de motifs, selectionnant un schema pertinent parmi differents schemas de confron- 
tation selon que chaque element gestuel, dans les motifs types, est statique ou dynamique, et confrontant 
lesdits motifs de langage des signes au motif de chaque element gestuel en utilisant le schema de confrontation 
selectionne, independamment des autres elements gestuels, et 

lesdits moyens de reconnaissance de morpheme integrant les resultats de la reconnaissance des elements 
gestuels constituant chacun un morpheme de langage des signes, sur la base d'une relation temporelle avec 
les resultats de la reconnaissance pour reconnaitre le morpheme de langage des signes. 



19 



EP 0 629 988 B1 



Systeme d'entree comportant : 

des moyens d'entree de langage des signes pour convertir une serie de gestes en des motifs gestuels qui 

sont des motifs serie de vecteurs caracteristiques et entrer les motifs gestuels convertis, 

des moyens de memorisation pour memorise r des motifs types associes a de gestes, 

des moyens de confrontation pour confronter les motifs gestuels qui ont ete entres aux motifs types de gestes 

memorises dans lesdits moyens de memorisation, afin de reconnaitre un geste exprime dans ledit motif qes- 

tuel, 

des moyens de reconnaissance de geste sensibles auxdits moyens de confrontation pour reconnaitre le geste 
exprime dans le motif gestuel, d'apres les resultats de la confrontation effectuee entre les motifs gestuels qui 
ont ete entres et les motifs types, et 

des moyens pour effectuer un traitement correspondant au geste reconnu, 

chacun desdits motifs types associe a un geste, memorises dans lesdits moyens de memorisation de motifs 
type, comportant une description permettant d'identifier si chacun des elements gestuels constituant un geste 
est un motif statique ou un motif dynamique, et un motif d'element gestuel exprime sous une forme qui differe 
selon que ('element gestuel est un motif statique ou un motif dynamique, 

lesdits moyens de confrontation pour confronter les motifs gestuels qui ont ete entres aux motifs types de 
gestes, se reportant auxdits motifs types dans lesdits moyens de memorisation, selectionnant un schema 
pertinent parmi differents schemas de confrontation selon que chacun des elements gestuels, dans les motifs 
types, est statique ou dynamique et confrontant lesdits motifs gestuels au motif de chaque element gestuel 
en utilisant le schema de confrontation selectionne, independamment des autres elements gestuels, et 
lesdits moyens de reconnaissance de geste integrant les resultats de la reconnaissance des elements gestuels 
constituant un geste, sur la base d'une relation temporelle avec les resultats de la reconnaissance. 

Systeme d'entree selon la revendication 9, dans lequel le motif type de geste est exprim6 par un vecteur carac- 
teristique dans le cas ou le geste possede un motif statique, et est exprime par un motif serie de vecteur caracte- 
ristique dans le cas ou le geste possede un motif dynamique. 

Systeme d'entree selon la revendication 9, dans lequel lesdits moyens de reconnaissance de gestes calculent le 
chevauchement des positions de reconnaissance (detection) entre elements gestuels reconnus (detectes) par la 
confrontation des motifs d'elements gestuels, et determinent qu'un geste correspondant a un motif type conf ronte 
au motif gestuel qui a ete entre est present lorsque le chevauchement calcule entre elements gestuels excede 
une valeur predeterminee. 

Systeme d'entree selon la revendication 11 , dans lequel, lorsque tous les elements gestuels constituant un geste 
sont des motifs statiques, lesdits moyens de reconnaissance de gestes determinent qu'un geste correspondant a 
un motif type conf ronte au motif gestuel qui a ete entre est present en reponse a la detection de I'element gestuel 
detecte pendant un intervalle de temps predetermine. 

Systeme d'entree selon la revendication 1 2, dans lequel le motif type de geste est exprime par un motif d'elements 
gestuels et par des informations annexees indicatives des donnees a extraire a partir du motif gestuel, et 

lesdits moyens de reconnaissance de gestes calculent et extraient les donnees necessaires a partir du motif 
gestuel sur la base du resuftat de la reconnaissance du geste et des informations annexees contenues dans 
le motif type. 
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